Automatic Explanation Spot Estimation Method Targeted at Text and Figures in Lecture Slides

Because of the spread of the Internet in recent years, e-learning, which is a form of learning through the Internet, has been used in school education. Many lecture videos delivered at The Open University of Japan show lecturers and lecture slides alternately. In such video style, it is hard to understand where on the slide the lecturer is explaining. In this paper, we examined methods to automatically estimate spots where the lecturer explains on the slide using lecture speech and slide data. This technology is expected to help learners to study the lectures. For itemized text slides, using DTW with word embedding based distance, we obtained higher estimation accuracy than a previous work. For slides containing figures, we estimated explanation spots using image classification results and text in the charts. In addition, we modified the lecture browsing system to indicate estimation results on slides, and investigated the usefulness of indicating explanation spots by subjective evaluation with the system.